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SiKty^nine amployers of rasaarchei'S and avaluators 
participated in a market rasaarch study conducted by the Departmant 
of Educational Foundations and Research at Gallaudat University, The 
sample includad consulting firms, go^^rnment agancies, professional 
associations, and other aducational agancies in tha greater 
Washington, D,C. area, as well as rasidential schools for tha daaf 
from around the country, A telephona interviaw inquired about i (1) 
the natura of the organiEation; (2) the kinds of skills naaded; (3) 
respondant's background; (4) intarest in bacomlng an intarnship sita; 
and (5) future job prospects « Forty-Height of these amployers also 
returnad a Skill Hating Form distributad by mail, while each typa of 
organisation emphasised certain skills^ the employars agreed on a 
ganarlc set of skills possassad by potantially amployable 
rasaarcher/avaluators including traditional rasaarch methodology, 
avaluation, administration/coBmunication, statistical analysis, and 
computer usage » Most (S9%) thought that contant specialty skills 
could ba picked up on the job. Psychometric skills ware Important 
primarily in educational settings. Of tha 69 employars, 62 eMprassad 
intarast in becoming an intarnship sita, and they ware very racaptive 
to hiring a haarlng-impairad intern. Raports of job prospects ware 
somewhat optimistic for consulting firms, privata corporations, and 
government agencies. An appendix includes all survey matarials, 
(LPG) 
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CAREER OPPORTIMITIES IN RESEAHCH AND EVALUATION 
RELATED TO HEARING 1I#AIHED PERSONS 



Donna H. Hermans 
Gallaudst Univarsity 
800 Florida Ava., N.E. 
Washlngl^on D.C. 20002 
202 651-5202 

What ara tha job prospects for haaring impaired individuals 
trained in rasaarch and avaliiation? I^at are tha job prospects 
for hearing individuals who ara trained in rasaarch and 
evaluation of hearing impaired populations? TOiat skills should 
be includad in a rasearah and evaluation training program? What 
other faetors need to be oonsidarad in a resaareh and evaluation 
program / suoh as intamship a^arianees? The answars to thasa 
guastions depend on tha souroa and seopa of tha Information basa. 
A vtriety of souroas hava baan tapped in an attampt to giva 
taaehers in this fiald guidanoa in the dasign of their programs. 
Numerous lists of oompatanoias have baan pt^lishad that wara 
based on the logioal analysis of resaaroh and evaluation tasks ^ 
as well as on the as^arienoes and inforaad opinions of tha aarly 
laadars in tha fiald of evaluation (Payne^ 1974? Sandars^ li7S; 
Soriven^ 1974; atufflabaam at al*^ 1971i Worthan, 1978)* 

A second approach to determining what should be taught in 
rasaaroh and evaluation training programs is to examine tha 
ooursa syllabi that are used in such ooursework* TOiila 
evaluation training programs tend to be in schools of education 
or psychology^ they can also be found in specific disciplines 
such as social work, health, and management sattings (May, 
Flaischer, Scheirer, ^ Cok, 1986). Davis (19fi6a) aditad a volume 
that contains an analysis of avaluation coursa syllabi from four 
disciplinary parspactivas-aducation, psychology, health, and 
businass and management-as well as from an interdisciplinary 
point of viaw, she reported that spacific disciplines diffar in 
their curricular amphasis, howavar, ^ere is increasing agraamant 
about the general topics avaluation courses should addrass 
(Davis, 1986b) . Much valuable infonuatien can ba obtained from 
this approach. However, additional information is nacassa^ to 
determine the correspondence between curricultm contant and the 
job market, as well as to identify variablas ralatad to 
internships and future job placuaant prospects. 

I wish to actaiowledga the important contribution that Steve Walk, 
Judy Harrison, Kate Tobin, and Judy Robbins playad in the data 
collection effort. The members of the Department of Educational 
Foundations and Research also made an important contribution by 
thair advica on tha dasign of ttia study. Finally, much 
appreciation is a^rassad for ttie afforts of the participants in 
tha study who gava willingly of tjiair time and idaas* 

PAPER PRESENTED AT THE 1987 AMIRICM^ EDUCATIONAL raSEARCH 
ASSDCIATieN raiETING, WASHINGTON, D.C. 
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Several researchers have investigated the skills that 
evaluators and their employers report as being neoeasa^ for 
suaeessful oonduot of research and evaluation studies. Two 
studies used this method to exMiine skills needed within the 
Gonte^rt of educational evaluations, Ma^ak^ Gray^ Mehrens^ £ 
Lezotte (1979) surveyed me:^ers of the American Educational 
Research Association's Division H who were primarily employed in 
educational settings. Worthen (1975) interviewed leaders of 
educational agencies that employ evaluators. Miderson and Ball 
(1979) surveyed a group of experts in evaluation from the fields 
of education^ health and social-action progrMas who were asked to 
rate the importance of a list of content areas and skills. 
Anderson and Ball reported their overall ratings ^ but they did 
not break dowi their results to reflect differences of opinion by 
the respondent's field of expertise. 

The present study was undertaken to detemine the views of 
employers from a variety of job placement settings and 
disciplinary areas concerning ttie critical skills for the 
research and evaluation profession^ implications for training^ 
and future job prospects. Several assmiptions ^ided ttim design 
of this study. First ^ employers of researchers and evaluators 
have a valuable contribution to make in terms of identifying the 
skills that are needed by practicing professionals in the .field. 
Second, a core of skills exists across disciplines, with 
particular emphasis within specific disciplines. Thus, the views 
©f employers should be examined within ttie context of the type of 
setting in which the evaluation and research occurs. Third, 
evaluators use the tools of research along with supplementary 
skills that are needed to fulfill the special reguirements of 
evaluation work. Fourth, individuals who are trained in 

evaluation are often called upon to function in the role of 
researcher, depending on the changing d^ands of the work 
environment, ^d, fifth, evaluation courses cam be taught from 
an interdisciplinary perspective (See Coimer, 1986) . 

In the current study, the DepartmMt of Educational 
Foundations and Research at Sallaudet Universitv conducted a 
market research study that was used as a t dim for the 
development of a degree prog«m in research and evaluation Uiat 
is consistent with fee existing job market. The general approach 
of the study was to sxirvey potential employers of the program's 
graduates to determine current staffing pattems, skills needed, 
potential internship placements, and anticipated job openings. 
Will toe results be of limited value because of Sailaudet's 
mission to serve Uie hearing impaired population? Three factors 
contribute to the generalizability of the findings. First, with 
the Increased implementation of mainstreuing in colleges and 
universities, it is increasingly likely that programs throughout 
the country will be serving students with some t^e of 
handicapping condition, such as hearing impairment* Second, one 
issue addressed in this study was the conduct of research and 
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evaluation with hearing ittpaired siabjsQts^ whi^ «?uuia amJm 
parallsl implioations for other raaearohare and ^TOiltiauQri; who 
are being trained to work with epsoial neads populafti^Jim, IhJFd, 
employers were asksd abput rassarchera and evaluatei^s J a g^nar& l , 
as wall as partiQularly about hearing impaired indl3Ti#uaii, 

KETHQDOMGY 

Subjects ^ Tha st^jaots in the study were potential #3iplfeyers 
of graduatas of a research and avaluation pr^g3r£.M wte wera 
salaotad from flva oatagoriesi oonsuiting £i3mm (irt^uding 
privata eorporations) , goveiOTient agencies, processional 
assooiations, rasidantial sohools for tee deaf, and other 
aduoational aganoias. Large residantial sohools for the deaf 
wara addad to tha list becausa Gallaudet eervas both hearing and 
hearing impairad students^ The initial phasa of the study 
involvad identifying thosa aganoies that aetually amployed 
avaluation researoh personnal to sarve as the sample. Exoapt for 
the residential schools, the search focused on opportunities in 
the greater Washington D.C^ area. 

Organizations Uiat possibly Miploy evaluation rasoaroh 
persoimel wara idantified by using the ISvaluation Network's 
vaoanoy list and list of organizations that employ evaluation 
rasaarohers; the C^P Yallow Pages listing of management 
aonsulting firms; the listing of professional associations in tha 
greater Washington D.C. area; directors of aduoational resaaroh 
in the school systams in Washington D*C, and in surrounding 
eountias; State Dapartaaent of Eduoation raprasentatives in MD and 
VA; aallaudat's Canter for Assassment and Demographic Studias* 
listing of large residential schools; and a listing of federal 
agancias that use evaluations, A computerised database was 
davalopad of the 288 organizations that potentially employ 
avaluation i^asaarch parsoimal. 

In ordar to identify organizations that actually do hire 
avaluation research parsonnal^ a short ^astionnaira (saa 
Qallaudat Collaga Response eard in the appandix) was sent in tha 
Spring of 1986 to thasa organizations. A follow-up mailing was 
sant to organisations ttiat did not respond to the first mailing 
within three waaks. Tha purpose of ttia first guastionnaire was 
to determina if toa organisations hired avaluators or 
resaarcharsy and if so^ if thay would be willing to participata 
in a follow-up intarv^iew. 

Tha rasults of ttiis first survey ara prasanted in Tabla 1. 
From the initial list of 288 nwaes^ a final sample was obtainad 
of 81 (28 percent) organizations that hira evaluation rasaarch 
parsonnel and who indioatad a willingnass to participata in the 
follow*up interview. Tha othar 207 naanas can be accounted for as 
baing undeliverabla (11 parcent) ^ not hiring evaluation rasaarch 
parsonnal (12 percant) , or not responding to the mailing (49 
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pB^Q^tit) * A 20 perasnt random sample of nonrespondente was 
aprit^Gtsd talephona^ The 2faeults of this £oll©w"Up are 

ptmB^mntmA in Table 2, The nonrespondants are primarily from 
a©sisailting ^Siras and profassional aesociations (76 paresnt) . 
Sisct^^eitfh't ^oaroant of the nonrsspondants did not return the 
qu%s^ionrtair^^ baoausa thay felt it did not apply to them* Given 
€liis infoMae^^^sion^ the large nonraeponse rata appears to be due 
primarily t© inaeeurately identifying the population, rathar than 
to a syetexaat^ie bias in tha nonraspondants . 

TASm 1 



FIRST SURVEY OF POTENTIAL EMPLOYERS 
OF EVALmTlON RESEARCH PERSONNEL 



^TiPl OF 




ORIGINAL 
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ThBlM 3 

TYPE OF 0RSAN12AT10NS INCLUDED IN THE SAMPli 



OF EMPLOYERS OF 


EVALUATION 


RESEARCH PERSOmTBL 




TYPE OF 
ORGANIEATION 


ORIGINAL 
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INTERVIEW 


ACTUAL 
INTERVIEW 
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10 
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8 
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20 
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Professional Associations 
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13 
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TOTAL 


81 


la 


€9 


48 



The types of erganizations in the sample are displayed in 
Table 3 * Residential sehools for the deaf were represented most 
frequently (40 percent) , followed by professional associations 
(22 peroent) ^ oonsulting firms (16 percent) , otoer sd.'icatiDnal 
agencies (12 percent)^ and government agencies (8 percent). (The 
other educational agencies included research departmenti in 
public schools^ State Departoents of Education, and imiversity* 
based research centers,) Twelve of ttie 81 sMiple members ware 
later unavailable to be interviewed. Therefore the aetual 
response rate for the telephone interviews was 85 perctnt (69 
respondents) . The indiv^lduals who were interviewed were asked to 
retum a form indicating their ratings of slcills nc^sdsd in 
evaluation research p Forty-eight individuals returaed this 
second form for a response rate of 59 percent on the skill rating 
form* 

The majority of the respondents (62 percent) in the 
telephone interview were trained in research and/or evaluatian 
methods* The other respondents were either trainsd in 

administration/personnel (13 percent), special/deaf education (9 
percent) , or in particular disciplines such as law, ttedicine, 
history, or English* Seventy-one percent of the respondsnts 
reported that they were actually doing research or evaluation 
themselves* These active researchers/ evaluators raportsd an 
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avarags ot 12.7 y%aM c=s£ expariancfe in the field. 

Pregeduyes ^ 5h# prooeduree for the idantif laation of the 
sampla , developmant eflC tha databasa, and tha initial survay of 
tha potential amploy^&f^e of evaluation rasaaroh personnal was 
desoribed in th^ pt^evious saotion. This saotion ineludaa a 
dBeoription of tha prdc^adures usad to obtain inforaation from 
those individuals Vh^ aonfinaed that thay did amploy evaluation 
resaarch parsonnal mnd ware willing to partiaipate in a follow-up 
intawriaw. 

Two questionnmit^ss wara davelopeds one for use in tha 
tslaphona intarviaw ari^a ona for rating possibla skills naadad in 
the evaluation teea^te-ah profession (sea tha appendix). Tha 
talephona interviaw ^^sstionnaira was dasigned to oolleot tha 
fQllowing inf oamationi 

-The nature of organization 

^Tha kind ef prograas/aGtivities ooeurring thare 
-The kind csf work that raseareh/avaluation paople do 
^Speoifi^ asc^Mnplas of evaluation/resaarah projaots 



*Tha kinds of ski&ls neadad 

-Whatoar o^ not it would be important to ba a oontent 

specialist^ and, if so^ in what areas 
-Any adaiti«3i=al skills tihat might be naadad 

"Tha raspOTdent*^ background 

-Itoethar of jaot tha respondent was trained in research 

or evaluation, and, if so, tha nature of that training 
-IWiether or isot the respondant was actively doing 

rasaarob evaluation, and, if so, for how many years 

-Intarast in parviiaipating as an internship site 
-Interast in iaving an intera 
-Process oj establishing an intarnship 
-Interast %n having a hearing impaired intern 
-Naed for w^n inte^ratar 

-^Future job pro^pteots 

-Job pro^pa^^^s at Uiair organisation 
-Job progpe^fee in lAa fiald in ganaral 

Tha talaphona Intter^iew questionnaire was pilot testad with 
threa organisations teimilar to those in the sampla. Tha 
intarviews were conflia^t^eo by three trained interviawars during 
Hay and June li86# Bs&li interview took about IS to 20 minutes to 
complete. 



Based ^ on evalaafe^ion • s relianca on traditional rasaarch 
methodologies and breadth of applications in rasaarch and 

ivaluation skills, % br^oad view was adoptad of the potantial 
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c^%^mmr paths for ttia pragram's graduatss, Thtrsfote ^ the 
f '^13- owing daseriptien wae fonaulatad of tte program intent* s 

Wa are planning a program to train people to 
be ©valuators or rasaaroh^rs who can go into 
an organization and help tham olarify thalr 
goals and determine how wall they ara meeting 
thosa goals* They will ba trainad to usa the 
soiantif io mathod to oollaot and analyze 
data, Thase skills oould ba applied in a 
variety of sattings such as in aahools to 
evaluate art individual's performanQs blm well 
as at tha program level. Thay ean also ba 
applied in market research^ in the 
astablishment of new programs or in pr^ojeete 
involving survey rasearoh, (Taken f^om tta 
Telephone Interview Questionnaire i sea the 
appendix « ) 

Tha skill rating questionnaire was davalopad baitd the 
folS3.owing souroas that were idantif iad in a oomputarlEed ^saarah 
of the literature ralated to avaluatien researah i^teill^s 
M^d^erson and Ball (1979) surveyed a group of e^erfcis in 
svaa^uation who were asked to rate the importanea of a l^st 
e©ji^S:ant areas and skills. Fienberg (1980) rsooamendad a fo^^us en 
leateming to use tha scientif io method for tha dasign of i^fcudies 
and lAe coliaestion and analysis of data t© BLmmmmm the sKt^^nt %^ 
whi^^h programs or aotivities met spaoified cbjeetivest DauS.ist^l 
attd Heddarson (1984) reoomandad a list of topie^ based on tteir 
avaEluation program in soeiolo^p Wortoan^ eairdray, an^S R#ls 
(IS^BO) based their list of skills on their progr^M 
Noy^thwestarn University. Maryak^ ^ray^ Mahrens, and L^zotta 
(1S'S79) surveyed mambers of tee ^erican Educational Ra^aarah 
Asp^^ciation • s Division H who wera primarily raplsy^^a in 
edtis^ational sattings. 

The skill rating guestiormaira was sent to tha 

indS^viduals who were later oontaotad in the talephona intar^^iaws* 
Th#^e individuals were raminded during the tslaphona intiiva.aw to 
ratt^m the rating form, Thay were also asked to idantify tta.a^ it 
ca^^^ from their organization, if Uiay wanted to. Nina tha 
ski^ll rating forms were raturaad wittaout suob idantifio^^tioJif 
qoji^eguently^ thay appear in Table 3 as "UnknoOTA" in refarac^aa to 
th© type of organization* 

_ miSULTS 

Kat^gre of tha Organization 

A rioh picture of tha organizations e»arged by askin^^ eattli 
re#^ppndant to describe tha organization's activitias, tha km^nd ^f 
worJ^c that evaluators/resaarchars did there, and spaelfic bh^mpI^b 
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prejeetP. Tli^ ^^flmlti^mg firms includsd private eorporationg 

^0 nil m.B itrtsS that eDnsulted for faderal and state 
t^VerMants, : pytv^'t# tusiii^es^ and non-profit and international 
^gf^nnisE* t/fow ©f work that wars dona ranged from 

p^c^spiuaisig^ mt^t^Mcal analysis^ data eollaetion^ raport 

wilting ^ lit^^armtUSE'* jfivlaw^^ program avaluation^ and conducting 
stjtvsyp^ t&i*% ^oi^#^ltiti©^s.s an rasaarah design* Sevaral 
iritarastiJig efpreji^ct^ ware raportadi (1) Survaying the 

p^ttieipaiits of a D#fertmenfc. of Agrioulture program using program 



^0^otiB to d%^t^ttftii^# mg^msm uaaga. (2) Analysing a coinpany'e 
w^tHorea t^e a#t^^3f^iiis t^.e nmfaar of minority amployees, (3) 
D#temins th^^ eiflp^n#^of raL^ing a child from infancy to oollega* 
(4) Oontraat Vltfc # eosipan^" to improva productivity and customer 
a^tiifaction* (5) Si^^wp Cor political candidatas to predict 
^J%ction out^^^»fe0, i^i, {6> marketing research for new products 
(ftoi food d%f^J^#^jt 

GovartC^eftt . ^^iliolas included fadaral^ stata and local 
5'#&rasentiitiv^a^^ |t th& Cadaral laval^ raspondents were 

iritsrviawsd fc^rea ^9#M^b tl^at sarva a variaty of functions such 
mg iupportiiimg %h0 P.S. Congraes^ disseminating govarnmant 
pp#&o):ts ^ and * csendxj^^^^^f tha cansus . State reprasantativas wara 
liifcOTiawed ' f afotf sxt^oision serviea and various aducational 

feiipiss * At 4 the kp^tl lavfel , the raspendante rapresentad social 
mewm agaft#^^i%0v Jht ^^^as of work that wara raported wara 
si^ilir^to ttl*^^^^ ^mp^M consulting fiwis^ with tha addition 

ot 4:^imly^l# ai aim important skill araa* Ixwnplas of 

prc^jiets lnel^Tae%f fc^Jlow'-u^ of progrm participants to detarmina 
l^i^gterm p^^^-<P^E*aitt #^fa(jts^ invastigating tha affect of a change 
xn thi drinkisaig €:^^fataLitias, assist devaloping eountriae in 

F0^iarch/%val*Ta^ti^ft jrojaetu*^ and detamina the appropriate 
al^ocition of ^%#^i*^c:i|g and ^taff needed to maat goals* 

Respond^aait^ pref e^sional associations rapresentad such 

diV'srs© ^QV:^pm iiploya^ent security agancias, lawyars^ music 

adiicitprs^ ce::3riraetl^3\jl off L-ciars ^ spacial educators^ physicians^ 
an4 ther^pig**^* Jb^aln^ ^^tt.a work dona by avaluation/rasaarch 
p^^iennal va^^ ^Is^ilji: to that reportad praviously^ with an 
a»M?'iiB on oittemb^^ship, ^hay collactad information on 

Pu^l toplQB a0 #^aiy Pavels in toe profassion^ budgat 

aiLoeationp, ^>^vial4fey .dproy^ams^ and reactions to naw products* 



Tha ra^^J.Qaii'tia,! loho&^s for ttim most part did not employ 
anyena who^a ©ol% t^^spens i^b i 1 i ty was to conduct rasaarch or 
avi.lyrtiQni Ih^^a ^iki wfe^^a saan as aithar integral in ttim job 
description f^^t psrs^ainal or wara saan as naadad but not 

PutJleiantly ^»a4r%^g^il und«3^ tha current staffing arrangements. 

inaje^^ity of tA^ re^^ondants dascribad tha individual 
agaiiimants ^t^dMi thaL-^ is done by tha school psychologist 

anJitally ap m^^^MpX^w ^ct ava^-iuation in thair schools* Howevar^ 
jQo^^ raspDnda«ifes imti^^iad evaluations of teachars and house 

pa>€nts' pasa^corti^^Q^i f^^low-up of graduates^ curriculum 
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svaluationS/ and surveys of parents and the daaf cenaunity. The 
work ^ that was mentionad by these rsspondants matchad that 
describad for the other types of organisations^ although tha 
emphasis was claarly on assessffient at tha individual level* 

In tha other aducational aganoies^ tha raspondents 
raprasanted offices of testing, resaarch and evaluation in public 
schools and on university oampuses. Tha kind of work mirrorad 
that raported in the othar organizations, with a greatar aaphasis 
on prograa avaluation, atudias wara eonductad in such areas as 
pravantion of high school drop outs, training school 
administrators to design evaluations, measuring minority 
achievement, building databasas of demographic and academic 
information at the local schools, davaloping consistent reporting 
mehhods, evaluating the gifted and talented program, and 
conducting research on child development, education and mental 
health in the area of deafness. 

Skills Needed 

Like Anderson and Ball (1979) , two of the respondents had 
some difficulty with completing the skill rating form* One 
indicated that skills are important in particular situations. 
Some of the skills are needed sometimes, but not all the time. 
However, when you do need the skill, it is critical* Another 
respondent was uncomfortable with the conceptual inequality of 
the items on the rating f om. He felt that some of the items 
represented full courses while others were more individual 
topics. These caveats should be recognized in Uie interpretation 
of the ratings. 

The respondents* ratings of the skills needed for research 
and evaluation are displayed in Table 4. Generally, all of the 
skills were rated as either essential or desirable by the 
respondents, thus indicating that the list of skills derived from 
the literature was "on target". Only advanced computer 

programming, time series design, observational research, and 
grantsmanship were rated as "not important" by more than 2 0 
percent of the respondents. Analysis of ratings by type of 
organization revealed a very consistent response pattern for all 
of the skill areas except in the psychometrics area. In this 
area, the rating "not important" was used by representatives of 
all of the types of organizations except one residential school 
and all of the other educational agencies. Thus, psychometrics 
appears to be a skill area that is emphasized more in educational 
settings than in the other work settings. 

Within each work setting and in each discipline, particular 
skills will receive greater emphasis. For example, Davis (1986b) 
pointed out that psychological evaluators would emphasize goal 
attainment scaling and management information systems. 
Educational evaluators would emphasiEe curriculim development and 
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school organiEation^ and businsss evaluators would eaphaeizs 
organizational change ^ market rssaarch, and cost bsnaf it 
analyeis. ^ile recognising thaee discipline specific skill 
areas, the skill ratings in the present study indicate that there 
is a generic set of skills that apply across t^es of 
organizations. These include skills in traditioiial research 
methodolo^, evaluation, adainistrat ion/communication, 

statistical analysis, and computer usage . 

In a paper entitled "Content Specialization and Educational 
Evaluationi A Necessary Marriage?", Worthen and Sanders (1984) 
examined the issue: Should an educational evaluator be trained 
as a content specialist, an evaluation specialist, or some 
combination of the two? They concl* ded that evaluation 

specialists are the best choice to evaluate most educational 
enterprises. They recognised that content specialization plays 
an important role in educational evaluation, but it is neither 
necessary nor desirable in the training of educational 
evaluators * 

The majority of the respondents (59 percent) agreed with 
Korthen and Sanders conclusion. They felt that general research 
and evaluation skills would be sufficient and that the person 
could "learn as they go". Fifteen percent of the respondents 
felt that some es^ertise would be necessary in such areas as 
administration, sign language, audiology, sociology, coiuiseling, 
the law, economics, agricultiire, criminal justice, mathematics, 
or reading* These respondents also felt that only some of the 
staff would need to be content specialists ^ One respondent 
recognized that the area of needed es^ertise would change based 
on what was being funded that year* The reaaaining respondents 
(21 percent) felt that e^ertise would be needed in such areas as 
engineering, militaa^ sciences, govarnment docmnents, foreign 
languages, reading, mathematics, deafness, sign language, 
sociology, health, the law, and finance. 

The respondents were asked to add any additional skills that 
they thought would be needed in th^jir particular setting to 
conduct evaluation and research studies* Over 60 different skill 
areas were listed r only a few of which were mentioned more than 
once. The residential schools' and two of the other educational 
agencies * represKitatives stressed toat manual coBmunication and 
]mowledge of deafness^ralated issues would be important. Four of 
the educational representatives felt that taiowledge of the 
subject area such as reading, mathematics, science, or social 
studies would be important. In addition, four of the educational 
representatives reported that experience in an educational 
setting would be important. 

Other areas ttiat were mentioned more than two or three times 
includedi legal issues, health, economics, personnel, finance, 
agriculture, French, Spanish, himan relations skills. 
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linguistiGS, CDunseling, SDciQlogy^ human developmerit , 
psychology^ audlology, and vieu^l impairments* 

intaraBt in an IntsmBhip 

Sixty-two of the 69 rsepondents reported that they would ba 
intsrasted in having an interti work in thair organization* The 
sama pareantaga raported an interest in having an intam who was 
hearing-impaired p One rapraeantativa a^^rasead some reservation 
about having a haaring impaired intarn bacausa much of their work 
is international and raquiras fluanoy in a foreign language. 
However, this representativa said that at tha antry level thare 
would be appropriata positions for individuals with evaluation 
research skills that did not raguira comiunication in a foraign 
languaga* She did indicata that sha thought it would be hard to 
advancs at that organization without thasa skills. 

Savaral important points came out of discuSBions about the 
internship process. First, tha raspondants revealed a vary 
positive attitude toward having a hearing impaired intarn. 
Sample commants included i 

"We try to da things batter than other psopla. X toiow that a lot 
of deaf people can do a let on their own," 

"Wa would ba aagar to do that (hire a haaring impaired intern) . 
We nead indepth study of mathods such as sapling, aggragating 
data,.," 

"We pride ourselvas on baing eclectic— baing abla to work with 
divarsa tj^es of people," 

The idea of having an intawi who was Jmowledgaable about daafness 
and rasaarch and avaluation appaaled to the respondents from the 
residential schools for tha daaf and from various social servica 
agencies. Sample eonmante includadi 

"Sand 'am up hera for practical es^erienee. We really do need 
someone, " 

"Daafness doesn't matter. Depends on the nature of tha task." 

"We work with hearing impaired clients, mo it would be a 
natural . 

"More than one would be welcome," 

Second, the respondents emphasiEed the need for prior 
planning and sufficient support resources, as exemplified by the 
following comments i 

"Hearing status is not important. Availability of an internship 
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is dependent on the match between the project and the 
gualif ications of the student." 

"We would be interested in hiring a hearing impaired intern, I 
am eancerned about eommunication and not giving the intern a 
passive role for six months." 

"You would need to interview the ccnsultantB on staff to 
determine an appropriate project for them." 

"I am oonoerned about ooraaunication and not giving the intern a 
passive role. You need to allow nine months to formally 
establish an internship with personnel and get a sponsor to help 
you push through the paperwork. " 



"We are very enthuBiastio about hiring an intern. We may work 
out room and board for 20 hours of work a week," 

These comments raise several important issueB. Firsts the 
respondents recogniEed the commmiioation problems faced by the 
hearing impaired intern. Except for the residential schools^ the 
majority of the respondents reported that aallaudet would need to 
furnish an interpreter to facilitate communication. (The one 
oral residential school respondent comiented that^ " They must be 
able to communicate orally and must promise never to use any sign 
language.") Second^ the respondents emphasiEed the need to find 
a match between t^e organisation's work and toe skills of the 
intern. This recpiires considerable planning and supervision of 
the internship placements to insure that the es^erience is 
meaningful to the student and that the student is making a 
meaningful contribution to the organization. Daudistel and 
Hedderson (1984) reconunend that the faculty visit the internship 
site to e^lain to toe agency toat toe trainees must do research 
and/ or evaluation work^ not clerical tasks. The intems need to 
obtain practical e^erience in applying research and evaluation 
skills; they need to collect useful data in toe context of an 
operating program without dismpting normal operations of the 
program. Thirds toe respondents seemed eager to learn about 
working with a hearing impaired person. Therefore^ some deaf 
awareness education might be needed to facilitate the process. 
Foiirth, toe issue of paid versus non^-paid internships needs to be 
worked out. Possible source of stipends need to be e^lored* 
Fifth, considerable pre-planning time is needed. 

Future Job Prospects 

In toe initial screening guestionnaire^ respondents were 
asked to indicate how many staff and consultants they e^^ected to 
hire in the next three years* This must have been- a difficult 
question for the respondents to answer because 44 percent left it 
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blank for the nuTOber of etaff aeiE^ers and 73 percent left it 
blank for the number of oonsultants* Of those who provided an 
answer^ 59 percent indicated that they planned to hire at least 
one staff member^ with an average response of a^fi people and a 
range between 1 and 15. Although fewer respondents answered the 
consultant question, 59 percent of those who did also indicated 
that they planned to hire at least one consultant, with an 
average response of 3.6 and a range between 1 and 20. 

In the^ telephone interview, respondents were asked to 
describe their view as to job prospects in their owi organization 
and in the field in general. Their responses provide some insight 
into why the question of future hirings was so difficult. 
Basically, the respondents indicated that future hirings were 
dependent on availability of funding, economic grotrth, and 
attrition, without knowing if there would be funding cut backs 
or who might be leaving and when, it was difficult for the 
respondents to specify how miiny new hires would be needed, 

BxK of the eleven representatives of professional 
associations reported that job prospects in their organisation 
were "slim" to "not good"* Thr others said that the chances were 
"fair'' to "good". However, Qnly two of the eleven professional 
association representatives thought that the job prospects in the 
field in general were "slim" or "not good". The others indicated 
that prospects would be good in larger fims, or very good 
because organizations that serve the public need to be 
accountable. 

Two of the nine consulting firms/private corporations 
representatives reported that they could not say for certain what 
toe job prospects were because they "fluctuate and are hard to 
determine". However, five of Uie oUier representatives described 
their situation pordtively, e.g., "growing", "veiy likely thmy 
will be hiring", "good", "a variety of needs exist", and "changes 
year to year, but IcDoks good now". Their responses followed a 
similar pattern with regard to toe field in general. 

The goveriment agency representatives painted a similarly 
positive picture, e.g., "20 to 30 jobs .a year", "need to expand", 
"continue to be important to •tell toeir story*", "high demand 
now", and "will be opportmiities". They also felt very "bullish" 
©n job prespeetii in toe field in general, seeing "more demand for 
skills to support progr^s". 

The representatives of the residential schools were about 
ei^ally split in describing the job prospects as good and poor. 
Most agreed that work needed to be done, but they wondared where 
the funds would come from. The one's who described the prospect 
as good based that on "an emphasis on long term planning", "need 
to document toe job they are doing", "providing services 
throughout the state", and "good on a consultant basis". The 
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reepsndents tthat thought job prospscts ware poor Qosmsntad that 
it was basically a problem of "a strong need but little money". 
Some did not see jobs for the "straight evaluator/researaher at 
the individual school level, but rather at the division or state 
level". Thus, the residential school representatives generally 
saw a more positive picture for job prospects in the field in 
general than in their individual schools. 

m other educational agencies, six of the eleven 
representative described job prospects positively, e.g., "expect 
to hire 3 to 10 people in the next 3 to 5 years", "moderately 
positive", "chances are good with research skills". Those with a 
more ^ negative outlook again blamed the problem on inadequate 
funding. Seven of the respondents indicated a positive feeling 
about job prospects in the field in general, with one person 
predicting "a s^stantial increase in the need for competent 
people in evaluation and research" . 

In the telephone interview, respondents were also asked to 
indicate whether or not they would hire a hearing impaired person 
at their organization to do research and evaluation. 
Interestingly, only 3 of tiie 69 representatives reported that 
they would not consider hiring a hearing impaired person* One 
person reported that he did not have enough business to keep 
himself busy not to mention an inters also, toother respondent 
said that their office was too small to accommodate an intern and 
an interpreter also. The third one said that they already used 
law interns and that was sufficient personnel. 

The respondents seemed to be more concerned with getting a 
qualified employee than with tiie person's hearing status. 
Bsveral of toe respondents raised issues similar to those 
discussed in relation to hiring a hearing impaired intern. They 
were concerned about the commtmication problems and who would 
provide an interpreter when it was necessary. Several indicated 
that they were "eager to hire" handicapped workers and that they 
had had hearing impaired people work for tora before* 

riummary and Conclusions 

The purpose of this study was to deteraine the views of 
employers from a variety of job placement settings and 
disciplines concerning the critical skills for the research and 
evaluation professional, implications for training, and future 
job prospects* Through a phone survey of 69 employers of 
evaluation research personnel and a mail survey of a subset of 48 
of those employers, inforoation was obtained regarding the nature 
of the organisation, the kinds of skills needed in research and 
evaluation, the respondent's background, interest in being an 
internship site, and future job prospects. The organisations 
surveyed represented consulting finoas, private corporations, 
govermient agencies, residential schools for the deaf, other 
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edueational agsnoies, and professional associations. 

Tha results of the survay aupport the position that it is 
poSBible to identify a generic set of skills needed to do 
rp- ^arch and evaluation that apply across a variety of types of 
^^.^izations* These include skills in traditional resaarch 
methodology, evaluation, administration/coffiinunication, 

statistical analysis, and coaputar usage* In some settings, 
particular skills would ba amphasized mora than in others. For 
axaapla, in residantial schools, toiowladge of daafnass and sign 
langua^a (excapt at an oral school) would be important. In some 
educational settings, knowledge of psychomatrics or spacific 
siibject areas such as mathamatios or reading would be important, 
Othar organizations 3'eported that it would ba desirable to have 
e^^artisa in such areas as social walfara, law, criminal justice, 
foreign languages, milita^, health, finance, personnal, and 
human relations skills. An interdisciplinary approach to 
teaching evaluation research would focus on the generic skills 
and allow enough flaKibility to address these spacific skill 
areas as well. 

Sixty*two of the 69 respondents reported that they would be 
interested in having an intern work in thair organization, 
whether tha intern was normally hearing or hearing impaired. 
Their eomments revealed a very positive attitude toward having a 
hearing impaired intern. The Idea of having an intern who was 
Imowledgeable i^out deafness and research and evaluation appealed 
to the respondents from the residential schools for the deaf and 
from various social service agencies. The respondents also 
stressed the need for prior planning and sufficient support 
services (in the way of interpreters) to insure Uiat the 
inteniship a^^erience is meaningful to toe student and that the 
student is ^ making a meaningful contribution to ttia organization. 
Education in deaf awareness may be needed in some organizations. 
The issue of paid versus non-paid internships needs to be worked 
out and possible sources of stipends need to be es^lored. 

Projecting job openings is a difficult task, given that such 
openings are dependent on the availability of funds, economic 
growth, and attrition. However, even wito this imcertainty, 59 
percent of the respondents who were willing to speculate on this 
reported that they ejected to be hiring evaluation research 
personnel in the next toraa years. Respondents in smaller 
professional associations ware not optimistic i^out future job 
prospects. However, those in larger f inns saw an increased need 
to be accountable to their public and thus were more optimistic 
in their outlook. Representative of the consulting firms, 
private colorations, and government agencies were generally 
positive with regard to future job prospects, e.g*, "changes year 
to year, but looks good now", "20 to 30 jobs a year", "need to 
e^and", and "more demand for skills to support programs". The 
representatives from the residential schools were mixed in their 
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opinians of future jab praspects* They agraad that the work 
needed to be dane^ but wandsrad where the funds would come from. 
Jab praspacts were aiare positive in sehaala with an emphasis an 
planning and dacmnentatian of their services. In the other 
edueatipnal agenoias^ tha picture was alsa mixed and was alsa 
blamed an the prablam of inadequate funds. Seven of the eleven 
respandants fram the edueatianal agenaies indioated a positive 
feeling about jab praspeats in the field in general. Overall^ 
the resul'cs ^ indicate a very receptive j ab market for bath 
normally hearing and hearing impaired evaluatars/researchers ar 
for evaluatars/researchers in the area of deafness* 

Are the results biased because the agency conducting the 
study educates bath namally hearing and hearing impaired 
students at the graduate level? The influence af such a bias 
might be demanstrated in a mare restricted range of skills that 
employers thought a hearing impaired persan cauld manage, ar in 
mare pessimistic prajectians regarding internships and job 
praspacts. Hawever, the list af skills that resulted carrespond 
fairly well ta thase reported by other researchers (a*g., Davis, 
19S6b; Anderson Ball, 197S| Ma^ak, Gray, Mehrens, & Lezotte, 
197S) * The prospects for inteniships and job placements were 
both very^ positive. This result may be more positive than would 
be found in attier regions af the country because the study 
focused an the greater Washingtyn^ DtC. area* The multitude af 
gaveriment agencies, prafessianal assaciatians, and consulting 
firms provide a mare haspitable climate for the research and 
©valuation professional. The results concerning internships and 
job placamants might be replicated in other ma jar matrapalitan 
areas with a similar acanamic structure. 
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APPENDIX 

GAUAUDET RESPONSE C^D 
SHIi RATING FORM 
TELEPHONE INTERVIEW Q'OTISTIONNAIMI 
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GALLAUDET COLLEGE 

RESPONSE CARD 



Dmpany/lnstitution: 



clephonc Numbers, 



Number and rype of individuals who use evaluaimn /research skills: 





ON^STAFF 


ON CONSULTANT BASIS 




Number 


Field of Experiise 


Number 


Field of Expert ise 


Docrora] 










Master's 










Bachdor*s 










Hired in '84 










Sotcci to hir€ m 
the nra S veaxs 











Sample fieldi of OT^mise: e%*aluation, cdycation, busineis. siatistici, computer progmmming. 
political science, public administration, psychology, anrhropologjs sociolog^s economics, other 
(please speci^O- 

S7ould vou be wiUing to participate in a 20»minute phone conversation to help us identm- specific e%'aluation 
fsfs^rrh skiik that you look for in individuals you hire? U Yes U No 

>lease fold the response card so that the return address is on the outside. Staple or tape closed. 

THANK YOUl 
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□ 
□ 


3, Dcmiptive itaiiiiici 
i Inferenml ilMisiies 


□ 


LI 


D 


5. CnneUiion mi itpmrn 


□ 


□ 


D 


i Multiviiiate lUtistici 


[J 


u 


U 


1, NnnpiraiTieiric itatistici 


□ 


u 


U 


0, Qii^lliy control 


□ 


D 


U 


% lifementiry compiiier programming 


□ 


□ 


D 


10, Advincd corflpuief proirimming 


0 


u 


□ 


II, Use of canned pfb|tanii 


□ 


u 


□ 


IJ, Use of micro cornputer ioflw3re 


YCl 


mm 


IIICS 




□ 


D 


u 


1. Tell fonsiruciion 


D 


□ 


□ 


2. Reliablliry 


U 


□ 




3. Validity 


□ 


□ 


□ 


i Appliciiion of t:iii (paper m\ pencil; sitiiii 








performance) 


□ 


□ 


CI 


5, Norm and critciion - referenced teits 


□ 




u 


i Seleciing a measurement imirument 


U 


□ 


u 


I Assesiiiig t measurement insifumeni 


U 


□ 


□ 


8, Inierpreiing ten reiults 


□ 


a 


□ 


p, Reactive concerns and unobiiruiive meisurei 



rhm fold thii form lo ikt ik ntm Mm u on tk oMl 
ItQpk or Upi dolU 

THANK YOU! 



5'^ 



INTERVIEW 8LJIDE FDR RESPDNDENTS 
This im CYDUR NA M E) 

(Ask to speak to the contact per^men* thmy mr*m not available, 

ask for* a good time call back. Note that time in pencil on 

this sheet, > 



ilf you. at^i 



able to talk to the eontart pet-soni) 



I am calling tr^om Ballaudet Col lege MnMr^ning an evaluation 
r*esear*ch pr*ogr*am that i*ie ar*e makinfl plans to develop, I believe 
you t^eceived a que^t ionnair*e in the mail 'Pr'om us, 

BRANCH 1 
VEe PEDPLE 

-thm person r^esponded to the fir^st quest ionnair^e and eaid that 
he/she would be willing to talk with um^ says) 

Vou r*esponded to out* -First questionnaire and indicated that you 
would be willing to talk with us about this a little bit more. 
Is now a good time to talk? 

ilt not 5 schedule a convenient time to call back and make a note 
it, 5 Cl-f yes^ go ^o 1*1). 

BLANK PEOPLE 

(If the permpn responded to the ^irst questionnaire and IcPt 
blank the indication of beinfl willing to talk with us^ mays) 
You responded to our "First questionnaire and I was wondering if 
you would be willing to talk to me a bit more about it now? 

itf not^ says ) 

What would be a more convenient time to call back? (flake a note). 



i, 1 BEBIN INTERVIEW 

I want to minimise the amount o"F time that I take 'From you and 
maKimi^e the outcome of our conversation^ so to begin with, I 
want to briefly share with you our ideas of who an 
evaluat or /researcher is and what such a person does. 

We are operating from the definition of an evaluator as a person 
who can help an organisation clarify its goals and determine how 
well the organisation is meeting those goals- An 
evaluat or /researcher uses the mcientific method to collect and 
analyse data that is used to help clarify goals and to assess the 
: eHtent to which a program or activity achieves its specified 

object ives. The eval uat or /researcher ^ s skills can be 
^applied in schools in terms of evaluating individual needs and 
i^v performance as well as at the program level. They can also be 
§|j:applied: in market research or in establishment of new programs or 

ERIC 



Could you give me an idM of the kind of pr-ogram/aet i vity your 
organisation is involved in? 



You nay you have lglgi<_thls_numbgt^ up f r-om thg__£lrst 
oueMtA^mnainsi individaalm i^ho do thi^ kind of woi-k. Tould you 
dSSeribe fOT- me the kind of Honk that these individuals do? 



Would you give me one or t*M© speeifi^ eHamplM of 
evmluation/researoh projert s/art i vit iee that are done at you 
or gani Eat i on? 



yould it be Important for an eval uat or /researcher to be a 
specialist in your organiEat ion? 

What content areas of spesial isat ion sMOuld you be looking 
<e, S-f reading, eTOnomire, healthy criminal Just iM, 
parent —ch i 1 d ) ? 



I sent you a lirt of skills that inisht be important for an 
evaluat or /researcher. Do you have that list in front of you? 

Please take a moment to scan these, ftre there any skills not 
lifted that you think should be? 



Uould you complete the rating scale for the skills and mail it to 
ni^7 ftlso, if you dw't mind, would you w^ite (the name of _the 
combany ) on it so that we won't need to bother you again? 
Thanks, 

A few last questions. Are you yourself trained in 
evaluation/ research? What was the nature of your training? 



Are you doing e\ aluat a on/research yourself? How many years have 
you been doing work in this area? 



Um think that pi-aetical enperienM is mti ±mpor*tmr%t pm-rt of^ 
bMoming a skilled eval uat©i-/t-esear*cher. We plan to -Pii^st teach 
th^ necMSar^y skills to our* etudente and then pr^ovide them with 
an inter^nship to obtain this practical eHper^iencep 



i 11^ th^ logistics eould be worked out in an aeeeptable fe^ay^ 
i±^m,^ supervision ^roiii a faculty member^ timeliness) do you 
think there would be interest in having an intern from our 
pregrani work in your orgariimat ion? (IF YES TO £-1, BO TO £. 
IF NO TD ED TD £.4). 

BRfiNpH B 

S, £ <IF YES TD 1 I > 

Could we come vieit your organisation and talk with you in more 
de^^il about what those logistics would be? 



£.3 (IF YES to Sp £, schedule a time to vi^it), 
YPU. 



STOP, THSNiK 



CIF ND to £-£, askr) 

What would the process be for establishing an internship? 
yho would we need to contact? 



Some of our students might be hearing impaired^ fts you know^ 
some hearing impaired people need an interpreter to function in 
the "hearing" world, yould your organisation be willing to 
consider hiring an hearing inipaired intern? 



If we provided the interpreterl 



£. W What do you see i 
eval uat ion/research pi 



I future job prospects for 

*so>is in the future at your organisat ion? 



Generally? 



Would your ©rganis^at ion be willing to consider hiring a hearing 
impaired graduate of our * prograwi? 
If an interpreter was provided? 
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THANK YOU VERY MUCH. 



£.4 CIF NO TO £.1, SflYi) . , ^ 

What do you s«e as futur-e Job pr-osp-ctB for* .val uat lon/r^ar^h 
pei-sons in th» futut-s at youi- ot-Bani=at io-i? SenBr-ally? 



^™ of our. student B miBht be hear-iVii impaiT-ed. As you kr»ow, 
lo^e hea°i«B impaired p.opl. ne«d an Int.^pr^eter- to Junction 
the "hear-ini" wor-ld. Would your- or-Bani =at ior, be willinB to 
consider- hiSini^ he.i-ina impaii-ed gr-aduat. of our- pi-OB^an,? 
If an interpi-eter- was pr-ovided? 



THANK YOU VERY MUCH. 
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